Atty. Docket No. CQ10211 

AMENDMENT UNDER 37 C.F.R. §1.111 
U.S. Application No. 10/728,777 

AMENDMENTS TO THE CLAIMS 

This listing of claims will replace all prior versions and listings of claims in the 

application: 

LISTING OF CLAIMS: 

1 . (Currently Amended) A method of summarizing a stream of ordered information, 

comprising: 

generating a similarity matrix for the stream of ordered information; 

decomposing the similarity matrix based on a probabilistic matrix factorization into a 
plurality of component matrices using non-negative matrix factorization ; 

determining, for each component matrix, a representative portion of the stream of ordered 
information; 

extracting the determined representative portions; and 

combining the extracted representative portions into a summary of the stream of ordered 
information. 

2. (Original) The method of claim 1, wherein the stream of ordered information 
comprises at least one of at least video information, audio information, still image information, 
and text information. 

3. (Original) The method of claim 1, wherein the stream of ordered information 
comprises a plurality of at least video information, audio information, still image information, 
and text information 
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4. (Original) The method of claim 1, wherein generating the similarity matrix for the 
stream of ordered information comprises: 

windowing the stream of ordered information; 

parameterizing the windowed stream of ordered information; and 

determining the similarity matrix of the parameterized windowed stream of ordered 
information. 

5. (Original) The method of claim 4, wherein the stream of ordered information 
comprises at least audio information, and parameterizing the stream of ordered information 
comprises parameterizing the stream of ordered audio information based on at least one of a 
STFT Fourier Transform, a Mel-Frequency Cepstral Coefficients Analysis, a spectrogram, a Fast 
Fourier Transform and wavelet decomposition. 

6. (Original) The method of claim 4, wherein the stream of ordered information 
comprises at least video information, and parameterizing the stream of ordered information 
comprises parameterizing the stream of ordered video information based on at least one of a 
histogram, ortho-normal projections, deriving a decimated image from DC coefficients of 
compression macroblocks and discrete cosine transforms. 

7. (Original) The method of claim 4, wherein the stream of ordered information 
comprises at least text information, and parameterizing the stream of ordered information 
comprises parameterizing the stream of ordered text information based on at least one of a 
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sentence, a paragraph, a meta-data information, a term-frequency inverse-document frequency 
information and part of speech information. 

8. (Original) The method of claim 1, further comprising determining a number of the 
component matrices of the ordered information based on a function. 

9. (Original) The method of claim 8, wherein determining the number of the component 
matrices of the ordered information based on a function comprises determining the number of the 
component matrices of the ordered information based on a probabilistic factorization rank of the 
similarity matrix for the ordered information. 

10. (Currently Amended) The method of claim 1 , A method of summarizing a stream 
of ordered information, comprising: 

generating a similarity matrix for the stream of ordered information; 
decomposing the similarity matrix based on a probabilistic matrix factorization into a 
plurality of component matrices; 

determining, for each component matrix, a representative portion of the stream of ordered 
information; 

extracting the determined representative portions; and 

combining the extracted representative portions into a summary of the stream of ordered 
information. 
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wherein decomposing the similarity matrix based on a probabilistic matrix factorization 

into the plurality of component matrices comprises decomposing a similarity matrix S into a 

plurality of component matrices A K such that: 

^(ij) = W(i,k)H(kj),and 
S = WH=Z4 . 

where: 

W(i,k) is an N x K matrix; 
H(k,j) is a K x N matrix; 

N is a number of separable portions of the stream of ordered 
information; 

K is a number of basis vectors; and 
k is an integer. 

1 1 . (Original) The method of claim 10, wherein determining, for each component matrix, 
a representative portion of the stream of ordered information comprises determining, for each 
component matrix, for each of a plurality of candidate excerpts from a subpart of the stream of 
ordered information that corresponds to that component matrix, a measure of how representative 
that candidate excerpt is. 
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12. (Original) The method of claim 11, wherein determining the measure of how 
representative that candidate excerpt is comprises determining an average within-class matrix 
A K (q,r) for that candidate excerpt. 

13. (Original) The method of claim 12, wherein determining the average within-class 

matrix A K (q,r) for that candidate excerpt comprises determining A K (q,r) as: 

I r N 

N(r-q) t^ti 

where: 

N is a number of separable portions of the corresponding subpart of the stream of ordered 
information; 

r is a ending portion of the excerpt; and 
q is an starting portion of the excerpt. 

14. (Original) The method of claim 13, wherein determining, for the plurality of 
candidate excerpts from the subpart of the stream of ordered information that corresponds to that 
component matrix, a measure of how representative that candidate excerpt is comprises 
determining the average within-class component matrix A K (q,r) for excerpts with all possible 
starting points r. 

15. (Original) The method of claim 14, wherein determining, for the plurality of 
candidate excerpts from the subpart of the stream of ordered information that corresponds to that 
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component matrix, a measure of how representative that candidate excerpt is comprises 
determining the average within-class component matrix A K (q,r) for excerpts with all possible 
starting points r, each excerpt having a fixed length L between the starting point r and the ending 
point q. 

16. (Original) The method of claim 14, wherein determining, for the plurality of 
candidate excerpts from the subpart of the stream of ordered information that corresponds to that 
component matrix, a measure of how representative that candidate excerpt is comprises 
determining the average within-class component matrix A K (q,r) for excerpts with all possible 
ending points q. 

17. (Original) The method of claim 14, wherein determining, for the plurality of 
candidate excerpts from the subpart of the stream of ordered information that corresponds to that 
component matrix, a measure of how representative that candidate excerpt is comprises 
determining the average within-class component matrix A K (q,r) for excerpts having a length l K 
for that component k, where the length l K is determined as: 

where L T is a total length for the combined excerpts. 
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18. (Original) The method of claim 11, wherein determining the measure of how 
representative that candidate excerpt is comprises determining, for a given component matrix 
A K , a score Q L {k) (i) for the i" 1 starting position of the k' h component as: 

1 i+L N 

ML m=i n=1 

where: 

N is a number of separable portions of the corresponding subpart of the stream of ordered 
information; 

i is a starting portion of the excerpt; and 

L is a length of the excerpt. 

19. (Original) The method of claim 18, wherein determining the score Q L {k) (i) for the 
i' h starting position of the k th component comprises determining a starting point q^ for the 
excerpt to be extracted from the k"' component. 

20. (Original) The method of claim 19, wherein determining the a starting point q L {k) for 
the excerpt to be extracted from the k lh component comprises finding the starting point q L {k) • 
that maximizes the score Q L {k) for the k ih component. 

21. (Canceled) 
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22. (Currently Amended) The method of claim [[21]] _J_, wherein decomposing the 
similarity matrix into the plurality of component matrices using non-negative matrix 
factorization comprises decomposing a similarity matrix S into a plurality of component matrices 
A k using non-negative factorization such that: 

A k (i,j) = W(i,k)H(k,j), and 
k=\ 

where: 

W(i,k) is an N x K matrix; 
H(kj) is a K x N matrix; 

N is a number of separable portions of the stream of ordered information; 
K is a number of basis vectors; and 
k is an integer. 

23. (Original) The method of claim 22, wherein generating the similarity matrix for the 
stream of ordered information comprises generating the similarity matrix S such that: 

L mF =E S(i,j)log(S b (iJ))-S b (i,j),and 

u 

S = S b = WH; 

where S b is a linear approximation of the similarity matrix S. 
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24. (Currently Amended) The method of claim 23, wherein generating the similarity 
matrix for the stream of ordered information further comprises determining a similarity value 
S(i,j) for each position (i,j) in the similarity matrix S as: 

S(i,j) = exp(d c (y i ,v J )-l)[[.]] 

where: 

v ( . and Vj are parameter vectors generated from the stream of ordered information for i' h 
and j th portions of the stream of ordered information; and 

d c is the cosine angle between the parameter vectors v t and v y . . 

25. (Currently Amended) A storage medium storing a set of program instructions 
executable on a data processing device and usable to summarize a stream of ordered information, 
the set of program instructions comprising: 

instructions for generating a similarity matrix for the stream of ordered information; 

instructions for decomposing the similarity matrix based on a probabilistic matrix 
factorization into a plurality of component matrices using non-negative matrix factorization ; 

instructions for determining, for each component matrix, a representative portion of the 
stream of ordered information; 

instructions for extracting the determined representative portions; and 
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instructions for combining the extracted representative portions into a summary of the 

stream of ordered information! 

wherein the storage medium is a computer-readable medium . 

26. (Original) The storage medium of claim 25, wherein the stream of ordered 
information comprises at least one of at least video information, audio information, still image 
information, and text information. 

27. (Original) The storage medium of claim 25, wherein the stream of ordered 
information comprises a plurality of at least video information, audio information, still image 
information, and text information 

28. (Original) The storage medium of claim 25, wherein generating the similarity matrix 
for the stream of ordered information comprises: 

instructions for windowing the stream of ordered information; 
instructions for parameterizing the windowed stream of ordered information; and 
instructions for determining the similarity matrix of the parameterized windowed stream 
of ordered information. 

29. (Original) The storage medium of claim 28, wherein the stream of ordered 
information comprises at least audio information, and the instructions for parameterizing the 
stream of ordered information comprise instructions for parameterizing the stream of ordered 
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audio information based on at least one of a STFT Fourier Transform, a Mel-Frequency Cepstral 
Coefficients Analysis, a spectrogram, a Fast Fourier Transform and wavelet decomposition. 

30. (Original) The storage medium of claim 28, wherein the stream of ordered 
information comprises at least video information, and the instructions for parameterizing the 
stream of ordered information comprise instructions for parameterizing the stream of ordered 
video information based on at least one of a histogram, ortho-normal projections, deriving a 
decimated image from DC coefficients of compression macroblocks and discrete cosine 
transforms. 

3 1 . (Original) The storage medium of claim 28, wherein the stream of ordered 
information comprises at least text information, and the instructions for parameterizing the 
stream of ordered information comprise instructions for parameterizing the stream of ordered 
text information based on at least one of a sentence, a paragraph, a meta-data information, a 
term-frequency inverse-document frequency information and part of speech information. 

32. (Original) The storage medium of claim 25, further comprising determining a 
number of the component matrices of the ordered information based on a function. 

33. (Original) The storage medium of claim 32, wherein the instructions for determining 
the number of the component matrices of the ordered information based on a function comprise 
instructions for determining the number of the component matrices of the ordered information 
based on a probabilistic factorization rank of the similarity matrix for the ordered information 
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34. (Currently Amended) The storage medium of claim 25, wherein the instructions 
for decomposing the similarity matrix based on a probabilistic matrix factorization into the 
plurality of component matrices comprise instructions for decomposing a similarity matrix S into 
a plurality of component matrices A k such that: 

A k (i,j) = W(i,k)H(k,j), and 

S = WH = Y A Lt 
k=\ 

where: 

W(i,k) is an N x K matrix; 
H(k,j )is a K x N matrix; 

N is a number of separable portions of the stream of ordered information; 
K is a number of basis vectors; and 
k is an integer. 

35. (Original) The storage medium of claim 34, wherein the instructions for determining, 
for each component matrix, a representative portion of the stream of ordered information 
comprise instructions for determining, for each component matrix, for each of a plurality of 
candidate excerpts from a subpart of the stream of ordered information that corresponds to that 
component matrix, a measure of how representative that candidate excerpt is. 
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36. (Original) The storage medium of claim 35, wherein the instructions for determining 
the measure of how representative that candidate excerpt is comprise instructions for determining 
an average within-class matrix Ak (q,r) for that candidate excerpt. 

37. (Original) The storage medium of claim 36, wherein the instructions for determining 
the average within-class matrix Ax(q,r) for that candidate excerpt comprise instructions for 
determining Aic(q,r) as: 

Ml - r)m WT)t t AAm ' ny - 

where: 

N is a number of separable portions of the corresponding subpart of the stream of ordered 
information; 

r is a ending portion of the excerpt; and 

q is an starting portion of the excerpt. 

38. (Original) The storage medium of claim 37, wherein the instructions for determining, 
for the plurality of candidate excerpts from the subpart of the stream of ordered information that 
corresponds to that component matrix, a measure of how representative that candidate excerpt is 
comprise instructions for determining the average within-class component matrix Ak (q, r) for 
excerpts with all possible starting points r. 
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39. (Original) The storage medium of claim 38, wherein the instructions for determining, 
for the plurality of candidate excerpts from the subpart of the stream of ordered information that 
corresponds to that component matrix, a measure of how representative that candidate excerpt is 
comprise instructions for determining the average within-class component matrix Aic(q,r) for 
excerpts with all possible starting points r, each excerpt having a fixed length L between the 
stalling point r and the ending point q. 

40. (Original) The storage medium of claim 39, wherein the instructions for determining, 
for the plurality of candidate excerpts from the subpart of the stream of ordered information that 
corresponds to that component matrix, a measure of how representative that candidate excerpt is 
comprise instructions for determining the average within-class component matrix Ax(q,r) for 
excerpts with all possible ending points q. 

41. (Original) The storage medium of claim 38, wherein the instructions for determining, 
for the plurality of candidate excerpts from the subpart of the stream of ordered information that 
corresponds to that component matrix, a measure of how representative that candidate excerpt is 
comprise instructions for determining the average within-class component matrix At:{q,r) for 
excerpts having a length l k for that component k, where the length l k is determined as: 

; -T Tu A AhJ) 

K - Lt i i *t,uW 

where L T is a total length for the combined excerpts. 
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42. (Original) The storage medium of claim 35, wherein the instructions for determining 
the measure of how representative that candidate excerpt is comprise instructions for 
determining, for a given component matrix A k , a score Q L w (i) for the i lh starting position of 
the k"' component as: 

1 i+L N 
ML m=i n=\ 

where: 

N is a number of separable portions of the corresponding subpart of the stream of ordered 
information; 

i is a starting portion of the excerpt; and 
L is a length of the excerpt. 

43. (Original) The storage medium of claim 42, wherein the instructions for determining 
the score Q L (k) (i) for the i th starting position of the k"' component comprise instructions for 
determining a starting point q^ for the excerpt to be extracted from the k th component. 

44. (Original) The storage medium of claim 43, wherein the instructions for determining 
the a starting point q L {k) for the excerpt to be extracted from the k' b component comprise 
instructions for finding the starting point q L {k) that maximizes the score Q L (k) for the k' h 
component. 
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45. (Canceled) 

46. (Currently Amended) The storage medium of claim [[45]] 25 , wherein the 
instructions for decomposing the similarity matrix into the plurality of component matrices using 
non-negative matrix factorization comprise instructions for decomposing a similarity matrix S 
into a plurality of component matrices A k using non-negative factorization such that: 

A k (i,j) = W(i,k)H(k,j),and 
SsWH = f l A k , 

k=\ 

where: 

W(i,k) is an N x K matrix; 
H(k,j) is a K x N matrix; 

N is a number of separable portions of the stream of ordered information; 
K is a number of basis vectors; and 
k is an integer. 

47. (Original) The storage medium of claim 46, wherein the instructions for generating 
the similarity matrix for the stream of ordered information comprise instructions for generating 
the similarity matrix S such that: 

W =Z S{i,j)\og{S b {Uj))-S b {i,i),and 

u 
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S = S b =WH 

where S b is a linear approximation of the similarity matrix S. 

48. (Original) The storage medium of claim 47, wherein the instructions for generating 
the similarity matrix for the stream of ordered information farther comprise instructions for 
determining a similarity value S(i,j) for each position (i,j) in the similarity matrix S as: 

S(i,J) = exp(d c (y„Vj)-l. 

where: 

V; and Vj are parameter vectors generated from the stream of ordered information for i"' 
and j' h portions of the stream of ordered information; and 

d c is the cosine angle between the parameter vectors v,. and Vj . 

49. (Currently Amended) A stream of ordered information summarizing system, 
comprising 

a similarity matrix determining circuit, routine or application that determines a similarity 
between two portions of the stream of ordered information and that stores the determined 
similarity into a similarity matrix; 

a probabilistic decomposition circuit, routine or application that decomposes the 
similarity matrix into a plurality of component matrices based on a probabilistic matrix 
factorization using non-negative matrix factorization ; 

an excerpt determining circuit, routine or application that determines, for each component 

matrix, a representative portion of the stream of ordered information; 
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an excerpt extracting circuit, routine or application that extracts the determined 
representative portions; and 

a summary generating circuit, routine or application that combines the extracted 
representative portions into a summary of the stream of ordered information. 

50. (Original) The stream of ordered information summarizing system of claim 49, 
further comprising: 

a windowing circuit, routine or application that windows the stream of ordered 
information; and 

a parameterization circuit, routine or application that parameterizes the windowed stream 
of ordered information; 

wherein the similarity matrix determining circuit, routine or application that determines 
the similarity between two portions of the windowed and parameterized stream of ordered 
information. 
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